Nonparametric variable selection and classification: The CATCH algorithm

In a nonparametric framework, we consider the problem of classifying a categorical response Y whose distribution depends on a vector of predictors X, where the coordinates Xj of X may be continuous, discrete, or categorical. To select the variables to be used for classification, we construct an algorithm which for each variable Xj computes an importance score sj to measure the strength of association of Xj with Y . The algorithm deletes Xj if sj falls below a certain threshold. It is shown in Monte Carlo simulations that the algorithm has a high probability of only selecting variables associated with Y . Moreover when this variable selection rule is used for dimension reduction prior to applying classification procedures, it improves the performance of these procedures. Our approach for computing importance scores is based on root Chisquare type statistics computed for randomly selected regions (tubes) of the sample space. The size and shape of the regions are adjusted iteratively and adaptively using the data to enhance the ability of the importance score to detect local relationships between the response and the predictors. These local scores are then averaged over the tubes to form a global importance score sj for variable Xj. When confounding and spurious associations are issues, the nonparametric importance score for variable Xj is computed conditionally by using tubes to restrict the other variables . We call this variable selection procedure CATCH (Categorical Adaptive Tube Covariate Hunting). We establish asymptotic properties, including consistency.